Fully Automatic Camera for Personalized Highlight Generation in Sporting Events.

Personally curated content in short-form video formats provides added value for participants and spectators but is often disregarded in lower-level events because it is too labor-intensive to create or is not recorded at all. Our smart sensor-driven tripod focuses on supplying a unified sensor and video solution to capture personalized highlights for participants in various sporting events with low computational and hardware costs. The relevant parts of the video for each participant are automatically determined by using the timestamps of his/her received sensor data. This is achieved through a customizable clipping mechanism that processes and optimizes both video and sensor data. The clipping mechanism is driven by sensing nearby signals of Adaptive Network Topology (ANT+) capable devices worn by the athletes that provide both locality information and identification. The device was deployed and tested in an amateur-level cycling race in which it provided clips with a detection rate of 92.9%. The associated sensor data were used to automatically extract peloton passages and report riders' positions on the course, as well as which participants were grouped together. Insights derived from sensor signals can be processed and published in real time, and an upload optimization scheme is proposed that can provide video clips for each rider a maximum of 5 min after the passage if video upload is enabled.

A machine learning approach for the classification of sports based on a coaches' perspective of environmental, individual and task requirements: A sports profile analysis.

Well-designed talent programmes in sports with a focus on talent identification, orientation, development, and transfer support the engagement of young individuals and the pursuit of elite performance. To facilitate these processes, an analysis of task, environmental and individual characteristics per sport is much needed. The aims of this study were to 1) analyse whether unique profiles per sport could be established by generic characteristics and 2) to discuss similarities and differences for the potential application in talent development and transfer. By means of a validated survey, 1247 coaches from 34 sports ranked 18 characteristics on importance to their sports (0 = not important - 10 = very important). To discriminate the responses per sport a Discriminant Analysis (DA) was carried out. To refine the DA-classification, Uniform Manifold Approximation and Projection (UMAP) with CatBoost classifier was performed. To test the performance of the CatBoost classifier-algorithm, a confusion-matrix was generated. The cross-validated DA showed that 70.2% of the coaches were correctly classified to their sport. The UMAP/CatBoost technique revealed 75.1% accuracy with correctly predicted responses per sport ranging from 18.2% (sailing) to 98.2% (soccer). With varying precision, the algorithm was able to differentiate sports by importance of its characteristics indicating similarities and differences per sport.

Scalable Semi-Automatic Annotation for Multi-Camera Person Tracking.

This paper proposes a generic methodology for the semi-automatic generation of reliable position annotations for evaluating multi-camera people-trackers on large video data sets. Most of the annotation data are automatically computed, by estimating a consensus tracking result from multiple existing trackers and people detectors and classifying it as either reliable or not. A small subset of the data, composed of tracks with insufficient reliability, is verified by a human using a simple binary decision task, a process faster than marking the correct person position. The proposed framework is generic and can handle additional trackers. We present results on a data set of $sim 6$ h captured by 4 cameras, featuring a person in a holiday flat, performing activities such as walking, cooking, eating, cleaning, and watching TV. When aiming for a tracking accuracy of 60 cm, 80% of all video frames are automatically annotated. The annotations for the remaining 20% of the frames were added after human verification of an automatically selected subset of data. This involved $sim 2.4$ h of manual labor. According to a subsequent comprehensive visual inspection to judge the annotation procedure, we found 99% of the automatically annotated frames to be correct. We provide guidelines on how to apply the proposed methodology to new data sets. We also provide an exploratory study for the multi-target case, applied on the existing and new benchmark video sequences.

Extrinsic Calibration of Camera Networks Based on Pedestrians.

In this paper, we propose a novel extrinsic calibration method for camera networks by analyzing tracks of pedestrians. First of all, we extract the center lines of walking persons by detecting their heads and feet in the camera images. We propose an easy and accurate method to estimate the 3D positions of the head and feet w.r.t. a local camera coordinate system from these center lines. We also propose a RANSAC-based orthogonal Procrustes approach to compute relative extrinsic parameters connecting the coordinate systems of cameras in a pairwise fashion. Finally, we refine the extrinsic calibration matrices using a method that minimizes the reprojection error. While existing state-of-the-art calibration methods explore epipolar geometry and use image positions directly, the proposed method first computes 3D positions per camera and then fuses the data. This results in simpler computations and a more flexible and accurate calibration method. Another advantage of our method is that it can also handle the case of persons walking along straight lines, which cannot be handled by most of the existing state-of-the-art calibration methods since all head and feet positions are co-planar. This situation often happens in real life.

Scalable Semi-Automatic Annotation for Multi-Camera Person Tracking.

This paper proposes a generic methodology for semi-automatic generation of reliable position annotations for evaluating multi-camera people-trackers on large video datasets. Most of the annotation data is computed automatically, by estimating a consensus tracking result from multiple existing trackers and people detectors and classifying it as either reliable or not. A small subset of the data, composed of tracks with insufficient reliability is verified by a human using a simple binary decision task, a process faster than marking the correct person position. The proposed framework is generic and can handle additional trackers. We present results on a dataset of approximately 6 hours captured by 4 cameras, featuring a person in a holiday flat, performing activities such as walking, cooking, eating, cleaning, and watching TV. When aiming for a tracking accuracy of 60cm, 80% of all video frames are automatically annotated. The annotations for the remaining 20% of the frames were added after human verification of an automatically selected subset of data. This involved about 2.4 hours of manual labour. According to a subsequent comprehensive visual inspection to judge the annotation procedure, we found 99% of the automatically annotated frames to be correct. We provide guidelines on how to apply the proposed methodology to new datasets. We also provide an exploratory study for the multi-target case, applied on existing and new benchmark video sequences.

Extrinsic Calibration of Camera Networks Using a Sphere.

In this paper, we propose a novel extrinsic calibration method for camera networks using a sphere as the calibration object. First of all, we propose an easy and accurate method to estimate the 3D positions of the sphere center w.r.t. the local camera coordinate system. Then, we propose to use orthogonal procrustes analysis to pairwise estimate the initial camera relative extrinsic parameters based on the aforementioned estimation of 3D positions. Finally, an optimization routine is applied to jointly refine the extrinsic parameters for all cameras. Compared to existing sphere-based 3D position estimators which need to trace and analyse the outline of the sphere projection in the image, the proposed method requires only very simple image processing: estimating the area and the center of mass of the sphere projection. Our results demonstrate that we can get a more accurate estimate of the extrinsic parameters compared to other sphere-based methods. While existing state-of-the-art calibration methods use point like features and epipolar geometry, the proposed method uses the sphere-based 3D position estimate. This results in simpler computations and a more flexible and accurate calibration method. Experimental results show that the proposed approach is accurate, robust, flexible and easy to use.

Human mobility monitoring in very low resolution visual sensor network.

This paper proposes an automated system for monitoring mobility patterns using a network of very low resolution visual sensors (30 × 30 pixels). The use of very low resolution sensors reduces privacy concern, cost, computation requirement and power consumption. The core of our proposed system is a robust people tracker that uses low resolution videos provided by the visual sensor network. The distributed processing architecture of our tracking system allows all image processing tasks to be done on the digital signal controller in each visual sensor. In this paper, we experimentally show that reliable tracking of people is possible using very low resolution imagery. We also compare the performance of our tracker against a state-of-the-art tracking method and show that our method outperforms. Moreover, the mobility statistics of tracks such as total distance traveled and average speed derived from trajectories are compared with those derived from ground truth given by Ultra-Wide Band sensors. The results of this comparison show that the trajectories from our system are accurate enough to obtain useful mobility statistics.